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Abstract 
This research is a partial report on a country-wide survey in Japan of tertiary-level expatriate 
(Holliday, 1992) English-language teachers’ beliefs and opinions about entrance examinations 
and their creation. Guided by grounded theory, this qualitative study highlights factors which 
constrain or facilitate changes to these tests and, drawing on participants’ experiences as well as 
literature on organizational practice in Japan, offers a number of recommendations that might be 
useful for expatriate faculty members to consider when they wish to make changes to their 
institution’s examinations, such as having a complete understanding of the purpose of an 
institution’s entrance examination, being able to demonstrate why changes to the test might be 
necessary, understanding one’s role on the test committee, and seeking help from like-minded 
colleagues in a position to effect change. These recommendations may also prove useful to other 
expatriate English-teaching faculty working alongside Japanese (local) (Holliday, 1992) faculty 
in host tertiary institutions worldwide, as well as those responsible for training English-as-a- 


Foreign-Language (EFL) teachers who expect to work abroad. 
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The purpose of this study, part of a larger country-wide survey of expatriate faculty beliefs about 
entrance examinations, was twofold: to determine those factors which constrain or promote 
change to the English portion of university entrance examinations in Japan, as well as to offer 
suggestions to expatriate educators seeking to change their institutions’ entrance examinations 
for what they believe to be the better. From qualitative data collection method and analysis 
methods, the voices of 12 expatriate tertiary English-teaching faculty members reveal perceived 
obstacles or facilitators of change to their institutions’ tests. A look at literature on Japanese 
institutional behavior offers suggestions to aid English educators in effecting improvements. For 
the purpose of this research, “expatriate” colleagues refers to those born and raised in countries 
other than the one they work in, while “local” colleagues refers to those born and raised in the 
country within which they work. 

When speaking of English entrance examinations generally, it seems that expatriate 
English-teaching colleagues tend to express concerns related to test validity and reliability, and 
even now in Japan it is not unusual to hear sentiments similar to those expressed to Leonard 


(1998) by J.D. Brown in an interview: 


Why is it that Japan has 300 exams or more? These exams are being made by people 
who don’t know what they’re doing, who say they don’t know what they’re doing. They 
are doing the best they can, but ultimately they don’t know what they’re doing. They are 


preparing tests that are haphazard and of unknown reliability and validity. (n.p.) 


Is there, on the one hand, an apparent acceptance by local (Holliday, 1992) (Japanese) colleagues 


of a seemingly random approach to test design? Are expatriate colleagues generally more critical 
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of entrance examinations and seeking therefore to change them? Or, do local English-teaching 
colleagues perceive these tests in different ways from their expatriate English-teaching 
colleagues from the outset? 

Two accounts, one of Brown (Leonard, 1998), and the other of Murphey (2004) 
illustrate the experiences of expatriate English instructors attempting to improve the validity — 
whether a test measures what it is supposed to (Harmer, 2005) - and reliability — whether test 
scores are consistent (Bachman & Palmer, 1996) - of English entrance examinations in Japan. 
Both indicated that their task was difficult and, for the most part, unwelcomed at Japanese 
tertiary institutions. In an attempt to solve problems related to validity and reliability, Brown 
recommended doing statistical analyses of the results each year since piloting test questions is a 
sensitive issue due to security (Leonard, 1998). In a later article, in which Brown described 
himself as “ever-persistent” in his desire to not only critique but also improve the situation in 
Japan, he recommended a detailed series of steps in four areas: test design strategies, test content 
strategies, logistical strategies, and interpretation strategies (Brown, 2000) In spite of these 
recommendations, he reported that he had been repeatedly criticized for his attempts to improve 
the quality of university English entrance examinations, and moreover that he was “ignorant of 
the Japanese perspective” (p. 2) and was accused of practicing “cultural imperialism” (ibid). 

Tim Murphey writes about his struggles as a tenured professor to change one 
university’s English entrance examination for what he believed to be the better. When he was 
finally offered a chance to chair the entrance examination committee, he said he would do so 
only on the provision that he be provided with examination data from previous years to analyze 
(Murphey, 2004). Using recommended statistical methods (Brown, 1996), Murphey found that a 


low number of discrete-point vocabulary and grammar questions fit the criteria for good 
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questions, as did a moderate amount of reading questions, yet a high a number of listening 
questions fit the criteria but were granted less weight than questions on other sections. Murphey 
reported the findings to his colleagues, yet no changes were made either to the test or score 
weightings. In the end, out of frustration, he resigned from that position. 

It appears that Brown and Murphy experienced dissatisfaction with the Japanese 
university English examination system, and although they have made attempts to make 
improvements as they deemed necessary, have been stymied in one way or another and, in the 
case of Brown, accused of not knowing the “Japanese way” of going about things. Because there 
are few other accounts of expatriate English-instructor voices in the English examination 
construction process, this exploratory research seeks to understand what roles foreign faculty 
play with regards to construction of their university’s English entrance examinations, what they 
believe about the examinations in general, and whether they have been successful at changing 


their university’s English entrance examinations. 


Literature review 

University Entrance Examinations: Claims and counterclaims 
It has been suggested that that declining student populations are forcing universities to re-think 
and perhaps relax their entrance requirements in order to keep their classes full (Kamiya, 2009; 
Kinmonth, 2005). However, although students may be able to enter non-top-tier schools (Mulvey, 
2001), for those who wish to enter universities with higher rankings, “examination hell” will 
continue to be a reality (Kinmonth, 2005; Mori, 2002; Takeuchi, 1997). 

Much has been written on the Japanese university entrance examination system with 


regards to its contribution to maintaining an individual university’s prestige (LoCastro, 1990; 
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Mori, 2002), advancing socio-economic status (Kariya & Dore, 2006; Sasaki, 2008; Sato, 2002; 
Takeuchi, 1997), reproducing social hierarchies (Frost, 1991 ; Seargeant, 2008; Takeuchi, 1997), 
and providing a needed source of revenue (Kinmonth, 2005). Research also exists on the 
proliferation of “shadow educational institutions”, or cram schools/juku, that support and are 
supported by this system (Blumenthal, 1992; Cook, 2012; McVeigh, 2006; Rohlen, 1980; Sasaki, 
2008; Stevenson & Baker, 1992; Tsukada, 1988). 

Studies also abound on the extensive influence of entrance examinations on the entire 
system of English education in Japan (Law, 1995; Matsumoto, 1994; Scholefield, 1997; Yoshida, 
2003), their detrimental effects on students’ motivation and ability to learn and use living 
English (Honna, 2005; Seargeant, 2008), the difficulties in reforming such a system (Doyon, 
2001) and the fact that these entrance examinations are generally not written by experts in testing 
(Aspinall, 2005). 

Criticism of the content of these examinations has also been the subject of research, 
particularly with regards to which “standard” of English is being tested, the use of archaic 
vocabulary and structures, and the overall difficulty of reading passages (Brown & Yamashita, 
1995; Kikuchi, 2006; LoCastro, 1990), as well as the types of tasks commonly occurring on 
examinations (Kikuchi, 2006). Examinations are also cited as a main reason for Japanese 
teachers’ of English (JTEs) continuing to use traditional, grammar-translation-based teaching 
practices (Cook, 2010; Sakui, 2004; Sato, 2002; Sato & Kleinsasser, 2004; Wada, 2002). 

Critics argue that many of these criticisms are no longer valid since an increasing 
number of university English examinations are calling for applicants to demonstrate more ability 
at solving task-based, higher-order problems, and show less ability to translate and solve 


discrete-item problems (Guest, 2000; Mulvey, 2001). They also assert that claims of examination 
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washback on the teaching of English in Japan have been greatly exaggerated, since teaching and 
textbooks appear not to be have been influenced by these innovations on tests (Mori, 2002; 
Mulvey, 2001). Innovative tasks, such as “... open ended explanation, translation, paraphrasing, 
and listening tasks, as well as short essay writing components” (Guest, 2008, p. 16) also appear 
to be attempts at improving test validity and reliability. Nevertheless, a number of respondents in 
this study mentioned validity and reliability largely being unaddressed at their instititutions 


(Cook, in press-b). 


Characteristics of Good Language Tests: Validity and Reliability 

According to Hughes (2003), a “(language) test is said to be valid if it measures accurately what 
we intend to measure” (p. 26). He recommends writing explicit specifications for the test and 
including a representative sample of the content of the test; using direct testing methods; and 
ensuring that scoring of responses relates directly to what is being tested (Hughes, 2003). 

But, are entrance examinations in Japan /anguage tests? If they are, then in theory they 
should, according to Hughes (2003), be one of four types: proficiency, achievement, diagnostic, 
or placement (p. 11). In this study, almost all expatriate English instructors held at least an M.A. 
or M.Ed., and 67% of them took courses in testing and assessment; thus it can be assumed that 
they have some degree of familiarity with these kinds of tests, as well as their purposes and 
construction, and would therefore believe that their university’s English entrance examination 
should fall into one or more of these pedagogically-oriented categories. 

However, perhaps none of these typologies can be applied to English entrance 
examinations in Japan, because such exams, as I have argued elsewhere, may be seen as serving, 


in addition to or as opposed to, pedagogical purposes (Cook, 2013). 
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Purpose of the Research 
This particular study forms part of a large, nationwide survey of expatriate English-teaching 
tertiary institution faculty and their participation in and beliefs about their institution’s entrance 
examinations. The main study examined expatriate ELT faculty member roles, opinions about, 
and attempts at reforming entrance examinations. This particular study focuses on two questions: 
1) What constrains and facilitates improvement by expatriate, English-teaching faculty 
to entrance examinations? 


2) If improvement is deemed necessary, how can it be accomplished most successfully? 


Data Collection and Analysis 
Data were collected first via online survey and then by follow-up open-ended interviews based 
on survey respondents’ answers. Due the wide-ranging nature of the project, an online survey 
was felt to be the most efficient for soliciting an initially large number of responses, and 
attaining the language of participants (Creswell, 2009). The questionnaire was available on 
SurveyMonkey from October 27, 2010 to January 31, 2011. Snowball sampling (Dornyei, 2003) 
was used largely through social networking websites and e-mail solicitations to friends, Japan 
Association for Language Teaching (JALT) chapters and special interest groups (SIGs). The 
questionnaire asked respondents to give demographic information, talk about their role in 
entrance examination creation at their institution, give general information about test design, and 
share their opinions and beliefs about English entrance examinations in general. Respondents 
agreeing to participate in a follow-up interview were contacted and interviewed via Skype or 


telephone. 
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Because not all participants answered all questions, thus limiting generalizability of 
results, qualitative methods of data analysis were deemed most appropriate. The strategy of 
inquiry for this study was largely grounded theory (Creswell, 2009). For this part of the study, 
Cresswell’s (2009) generic guide for analysis and interpretation was followed. Data were 
collected and prepared for analysis and audio-recordings were transcribed. Data were then read 
through to gain a general impression of ideas and tone, and typed cases were prepared for all 
participants. Using NVivo qualitative coding software, material was organized into chunks 
(words or groups of words referring to a theme), and coding processes were used to generate 
themes for analysis (Bringer, Johnston, & Brackenridge, 2004). Finally, emergent themes were 
identified. The Appendix includes a list of questions that respondents addressed for this part of 
the study. 

In order to protect the anonymity of participants, pseudonyms have been used. As Table 1 
indicates, all come from British/Australasian/North American (BANA) countries (Holliday, 


1994) and all but two (Debbie and Mauve) are male. 


Table 1 


Participants 


Pseudonym Nationality Years teaching Type of 


in Japan University 
Arthur American 6-10 National 
Carlos Canadian 6-10 Private 
Debbie American 21-30 National 
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Diogenes American 31-40 Private 
Jack Australian 1-5 National 
James American 21-30 Private 
Joe American 11-20 National 
Mauve American 6-10 Private 
Mike British 11-20 Private 
Paul British 21-30 Private 
Peter British 6-10 Public 
Phil American 1-5 National 
Results 


Before presenting the results, it is important to state that this research highlights the beliefs and 


opinions of expatriate foreign faculty and that, perhaps due to factors such as the implicit nature 


of Japanese culture and the positioning of expatriate foreign faculty as soto muki (outward 


facing) in their universities (Poole, 2010), as well as the fact that the majority of them have not 


been raised or educated in Japan or Japanese institutions, it may be that they don’t have the same 


access to information as their local colleagues do. As a result, the cultural/educational 


backgrounds of expatriates as well as the possibility of their having an incomplete knowledge of 


all circumstances surrounding the system of entrance examinations in Japan, may influence their 


beliefs and opinions (Cook, in press-a). 


Factors inhibiting improvement 


Hierarchical structures 
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A number of respondents said that attempts to improve tests (for example, by doing statistical 
analysis on items) was difficult because they felt those in senior positions disagreed with change. 
James, for instance, felt that decisions about test-making “have to do with power and who has 
power within the university.” Mauve felt that “seniority and personal power trump expertise”, 
thus older faculty members, who in her words are “set in their ways... prefer the way things have 
always been done.” Diogenes said he’d heard “some comments from some people who were not 
much listened to ... because they’re younger and radical.” Thus, tests are unlikely to be 


improved unless initiatives are taken by people at the higher end of the faculty hierarchy. 


Face 

The potential loss of face for those at the top of the hierarchy if innovations were implemented 
on their examinations was another factor precluding change. For Diogenes, this loss of face was 
an issue that came up when the discussion turned to introducing statistical analysis to entrance 


examinations at an institution at which he was formerly employed: 


I think there were the same social pressures not to stand out... I think the same thing is 
affecting the introduction of statistics, looking at examination statistics, because some 
people are very much into this and they’ve done a lot of preparation and they’ve done 
statistics at school and they’ve done research papers involved with statistics, and other 
people don’t know the first thing about it. And so, there’s a great difference in level of 
sophistication of what this is all about. And as it happens, the people who are higher up 
in administration are the ones who don’t know much about this sort of thing. It’s a face- 


saving device, I think. 
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Fear 
In addition to hierarchy and face, for Jack, fear was another factor that prevented change from 
happening to examinations at his institution due to the possibility of problems occurring as a 


result of reforms to examinations: 


... it (the test) has to go through other people who may be non-academics or non- 
practicing teachers who have their own situation and their own position to protect, and 
one way they do this is relying on what they consider to be a normal practice... and 
these people have a fair amount of clout... and nobody wants to lose face... it’s a big 
hassle to try to persuade these other people to change their views and if they’re 
changing, they’re moving into something potentially dangerous because it could go 


wrong... 


Concerns about continuity 

One concern raised by several respondents was whether an innovation would persist after its 
initial implementation, especially after the originator left the committee or institution. At Mike’s 
university, critical thinking questions he’d introduced were initially taken on board favourably, 
but abandoned once his mandate on the committee was over. Whether or not to continue using a 
new type of question introduced by a Japanese colleague was also the subject of some discussion 
at Debbie’s institution: “We talked about it last year, whether or not to keep it, because this 


colleague retired... then somebody had to take on bringing in that question.” 
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No need for change 

Responding to the question, “Have innovations been introduced into your university’s English 
entrance examination in the last 10 years”, almost 60% percent of respondents said, “No”. The 
most common answer for this included “No perceived need.” One anonymous respondent wrote, 
“Change is difficult when there is no apparent problem with the existing format.” For Mike, this 
comes down to expatriate faculty feeling a need to make improvements to existing entrance 


examinations, while such a need is not felt, in his view, by Japanese colleagues: 


... Native speakers... because this is kind of their job to be on the committee, they have 
a sense that, “I should do this well, as I’ve been trained.” Whereas the Japanese teachers 
don’t see that as being necessary and objectively they are right. These changes are not 


necessary under present conditions. That’s simply a fact. 


Factors Facilitating Change 

Consensus 

Consensus was one oft-cited factor facilitating innovation on entrance examinations. On the on- 
line survey, after “expatriate faculty”, “the examination committee” was listed as being 
responsible for innovation on entrance examinations. According to Phil, “Everyone in our 
department adds something, adds some small part or [has] gradually been changing how the test 
has been done over several years.” Joe, in speaking of innovation on his institution’s examination, 
uses the words “we” and “our” claiming the group’s ownership in decision-making: “We’re 


always looking to change it mainly so that jukus (cram schools) can’t get too much of a hold on 


what our entrance exam is like.” 
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An opening 
Carlos revealed how he was offered a chance by the examination committee to create test items 
as he deemed appropriate. Because that offer was accepted, his term on the committee was 


satisfactory for him: 


... It’s not like they asked me to make the test better; it’s just that they said, ‘Okay, 
you’re on the committee. What would you like to do?’ And I put up my hand and said, 
‘I'll do the reading section if you don’t mind’... I came in the next week and the writing 
section wasn’t really done yet, so I said, ‘Well, I'll start on this as well, if you don’t 


mind’... 


However, related to the previous inhibiting factor of continuity, he said, “On the other hand, if I 


hadn’t done that, I’m sure nothing would have changed.” 


Hierarchical structures 

As was shown previously, although hierarchical structures can constrain innovation, they can 
also facilitate it. At Mauve’s institution, some senior committee members “...are open to 
innovation, and if they are persuasive enough - and have no real enemies on the committee - 
occasionally things do change.” At Joe’s institution, committee members at the top of the 
hierarchy were content to hand the reins over to junior members who were knowledgeable about 


testing. He wrote: 
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(Innovation) was felt necessary because the older tests were discrete-point and a few of 
the younger teachers (both local and expatriate) who had a good grasp of applied 
linguistics and testing theory were responsible for introducing the innovation... Some of 
the older teachers are obligated to be on the committee, but they don’t want to do the 
work. And that’s fine. They take a back-seat role... they don’t want the hassle that 


comes with it, so it works. 


Discussion 
With regards to factors constraining innovation, respondents seem to understand correctly that 
hierarchical structures prevail in Japanese organizations and that “older members have more 
power, earn more, and are considered more knowledgeable, even if this is not really the case” 
(Haghirian, 2010, p. 14), and, thus, have a say in whether or not innovation is to occur. Group 
harmony forces Japanese people to save, not only their own, but the face of others (Alston & 
Takei, 2005) as well, and fear may come from the fact that groups try to avoid risk in order to 
protect that face; by doing so, however, group members feel more secure (Haghirian, 2010, p. 
80). 

Regarding factors facilitating innovation, as pointed out by respondents in this study, 
consensus may be the single most important factor in effecting change. According to Haghirian 
(2010), it “is the reason why processes, once established, will hardly be individually improved or 
changed, even if a process proves inefficient or even damaging to the firm. Without group 


consensus, nothing can be changed... (p. 100). 


The Kyoto JALT Review, 1, 17-44 


31 | Cook: You Say You Want a Revolution? Changing Japanese 
University Entrance Examinations 


Recommendations 
In light of the above, what can be done by expatriate ELT faculty members who would like to 
make changes to their institutions’ entrance examinations? The following section offers a 
possible list of questions expatriate entrance examination committee members might want to ask 


themselves before embarking on a course of innovation. 


1. What is the Purpose of your Institution’s Entrance Examination? 

The first order of business is to find out if the entrance examination is, in fact, considered to be a 
language test. If the majority of local colleagues believe it is something other than a language 
test, such as a tool for social stratification, an indication of students’ general intelligence, proof 
of students’ ability to take tests, a tool for public relations, or a source of funding, then expatriate 
faculty members may have to work harder to make a case for change. However, if colleagues do 
believe it is a language test, the task might be easier. However, given the secrecy and sensitivity 
surrounding entrance examinations in general, this may be difficult to ascertain from local 


colleagues and may have to be handled delicately. 


2. Why do you Feel a Need to Change your Test? 
Mike sees no clear benefit to changing his university’s English entrance examination. Although 
he acknowledges the general benefits of good testing, he can’t see that his university would be 


advantaged in any way by changing its test: 


I don’t disagree with J.D. Brown or Tim Murphey in the sense that if there’s a better test, 


that’s surely a good thing, but there isn’t an advantage to a better test. What would be 
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the advantage? The only thing you could potentially gain would be if your current 
system is not getting the kind of students that you want to have and you think you can 
make a test that will get them. But you won’t be able to do that, because people won’t 
try to take your exam if they think your university... doesn’t have the status or whatever 
it is, ... so making a better test has no meaning that I can see outside of just feeling good 


that we’ve got a test that measures something. 


In other words, we need to ask ourselves if our tests are already doing what they should 


be doing and if they are, whether they really need to change. 


3. What is the Expatriate’s Role on the Test Committee? 
Jack told me he felt largely excluded from decision-making processes at his institution generally, 
and felt very frustrated because no one, local and expatriate teachers included, seemed to be 


listening to him: 


I was told that I would have to look at materials which other people were making... we 
ended up with about 80 pages of stuff including all the reading texts and everything ... 
and they kept asking me “Is this correct? Is this correct? And I said, “Yeah, but if you 
also look at the questions, the questions don’t relate to the text at all”. You can’t have 
questions asking about the phonemes for a written text. There’s no connection. And 
somebody looked at me [and said] “This is what we always do.” And I said, “I’m on the 


committee now, and I’m telling you that this makes no sense.” 


The Kyoto JALT Review, 1, 17-44 


33 | Cook: You Say You Want a Revolution? Changing Japanese 
University Entrance Examinations 


During our interview as I sought to make clear Jack’s roles and responsibilities on the entrance 
examination committee, I realized that he was acting as a proofreader, but he felt that as a 
committee member he was entitled to express any views about the test he had. 

In “Values in English language teaching”, Johnston (2003) writes about a fundamental 
tension in ELT professionalism: that although many teachers strive to be professionals, their 
aspirations are not reflected in the way their work is perceived by those around them. In other 
words, there is a disjuncture between what can be called claimed identity and assigned identity 
(p. 108). 

Jack, disclaiming his assigned identity on the entrance examination committee, was 
perhaps unknowingly doing himself a disservice. By not accepting his assigned identity and 
following Japanese custom, he might have been pitting himself against his colleagues, thus 


ensuring that his voice would not be heard. According to Haghirian (2010): 


Being successful in a Japanese team means being more passive and listening to older 
peers. In the context of a Japanese firm, being active in finding a role and taking on an 
independent task is considered very inappropriate in general and unheard of for 
beginners. This behaviour often leads to intercultural conflict (p. 101). 

While it may not be appropriate to call Jack a “beginner”, his status as “native speaker checker” 


might have been viewed quite differently from his colleagues’ role as question creators. 


4. Who can Help Make the Case for Change? 


It is unlikely that individuals can effect change on their own in Japanese institutions. 
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Haghirian (2010) tells the story of Laura Kriska, an American woman who worked for a 
Japanese automobile company and wished to make a case for the company to abandon a rule 
prescribing that only female employees wear uniforms. Kriska was advised to form a quality 
circle and with this group, was successful in obtaining a policy change by the company. 

As mentioned previously, consensus is very important; rarely in the Japanese context; are 
suggestions by single individuals accepted easily. Thus, if expatriate faculty want to make 
changes to examinations, they might want to enlist the help of other like-minded colleagues and 
make their own quality circles. Because hierarchy is also key, proposals might more likely be 
accepted if some members of the quality circle are higher up in the hierarchy and have decision- 


making power. 


5. How can Others be Convinced that Change is Needed? 

Convincing others that change is needed is also important. Making negative comments about 
existing formats may cause colleagues to lose face; thus, it is recommended that a concrete and 
well-reasoned plan of change is made. Carlos, for example, was able to implement changes, as 
mentioned above, because he went to his committee with clear pedagogical reasons for 
modification. Another strategy is to appeal to the face of the university in some way since public 


image and reputation are paramount. As Mike suggests: 


There’s something to be said, just sheer pride and professionalism to create a really 
good test, and that’s the best way to pitch it. If you just say, “Look, we should do it 
because universities in Japan have a bad reputation and are not in the top 100 in the 


world, so here’s something we can do to make it more professional and better.” 
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This point about finding a positive spin for the change is critical. Jack, who told his committee 
that their questions didn’t “make sense” may have chosen an unfortunate tactic. Expressing 
disagreement in a negative manner may be considered a violation of group harmony by local 
colleagues, which may have resulted in his perceived rejection by the group (Alston & Takei, 


2005). 


6. Who will Share of the Burden of Change? 

Although Carlos was able to successfully implement changes to his institution’s entrance 
examination, he said that had he not offered to do so, the test would have been in his words, the 
“same old, same old”, meaning that those who seek to implement changes will generally be 
responsible for them over the short or long term. Debbie and Mike earlier brought up the issue of 
continuity of innovation. At Debbie’s institution, it was decided that the new type of question 
introduced by a previous committee member was too good to abandon, and so responsibility for 
it was passed on to another committee member, yet at Mike’s, as he said, the innovation was 
abandoned once he was no longer on the committee. Thus, expatriate ELT faculty have to ask 
themselves, when they are asking for change, if their proposals will be accepted only during their 
tenure on the committee or for a longer period of time. If change is implemented, can the burden 
of such innovations be placed on colleagues when some expatriates are replaced on committees? 
And, as Jack’s case shows, can changes be asked for if expatriates are not responsible for 


creating questions in the first place? 


7. How Long are Expatriate ELT Faculty Members Willing to Wait for Change? 
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Peter, in discussing change to his institution’s entrance examination felt that gentle persuasion 
would be required as well as time. He is correct in that change does not occur quickly. In a book 


about Japanese business practices (Alston & Takei, 2005), the authors state that: 


Because of their fear of failure, Japanese are content with small victories. When change 
takes place in Japan, it generally takes place in small steps after careful study and 
consensus building. This of course takes time but fits the Japanese risk-averse 
preference.... Foreigners who wish to do business with Japanese should offer smaller 
proposals at first. The proposals should also be presented as continuation of the past 
rather than an abrupt break from tradition.... It is always easier to add to a policy than to 


ask Japanese managers to consider adopting new policies. (p. 89). 


Thus, expatriate entrance examination committee members will need to offer suggestions in 
small doses, acknowledge past formats as they seek to change present ones, take a long-term 


view, and will need to be patient. 


Conclusion 
Although a one-size-fits-all recommendation for change to entrance examinations in Japan or 
other countries is beyond the scope of this paper, it is hoped that the above will be helpful to 
those seeking to improve entrance examinations at their institutions. Like many expatriate 
English-teaching faculty living in Japan, this author has more questions than answers and is still 
struggling to find ways to improve the quality of her work. While this research project has 


yielded a great deal of information and has helped her understand more about the lives of 
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expatriate English-teaching faculty, and thus feel a stronger part of that community, these 
findings are the tip of the iceberg. Indeed, they may be helpful not only to expatriate English- 
teaching faculty in Japan, but to any expatriate faculty members responsible for test creation in 
any country, since the recommendations mentioned here might be useful to expatriates seeking to 
improve language tests that might have a long history and that might be used differently from 
those in the expatriate’s country of origin. 

The results of this study may also be useful for those teaching graduate students about 
testing and assessment. In addition to teaching content about testing and what constitutes good 
tests, teacher educators may also turn a critical eye to their beliefs and highlight those aspects, 
which, while appropriate in one culture, may have limited applicability or suitability in another. 

Lastly, an obvious recommendation for future research is to conduct a similar study 
with local English-teaching faculty to see if there is any correspondence between their opinions 
and those of their expatriate colleagues. Such an insider view of entrance examinations-would 
invaluable to expatriate colleagues who would benefit from their experience and expertise. 

Throughout this research project, my own knowledge of and beliefs about the purposes of 
English on entrance examinations at Japanese universities has been challenged considerably and 
although I can’t help but continue to advocate for the construction of valid and reliable tests, I 
realize that I must not position myself as an “expert” on testing on the grounds of having taken a 
few courses on assessment in graduate school. I must work with my local colleagues to make the 
best tests possible for the purposes my institution deems appropriate and, where I see a case for 
reform, make it in such a way that its benefits are perceived by all stakeholders. Although the 
tradition of testing may seem like an impenetrable barrier, I believe local colleagues are 


interested in what expatriate colleagues have to say, and that test improvement, where warranted, 


The Kyoto JALT Review, 1, 17-44 


38 | Cook: You Say You Want a Revolution? Changing Japanese 
University Entrance Examinations 


is possible. 
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Questions addressed by respondents related to this study 


Section Questions 
Innovation on your university’s ° Have innovations (paragraph writing, etc.) been 
entrance examination. introduced into your university’s English entrance 


examination in the last 10 years? 

o (If “yes”) What kind of innovations were 
introduced? 

° Why was it felt that an innovation was 
necessary? 

° Who was responsible for introducing the 
innovation? 

° Was the introduction of the innovation 
successful and permanent? 

o (If “no”) Why haven’t innovations been 


introduced? 


Overall opinions e Overall, do you agree that your university’s 


entrance examination is an effective measure for 
entrance examination is an effective measure for 


selecting students to attend your university? 
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° What do you believe would improve the 
effectiveness of your university’s entrance 


examination? 
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